The automatic and unsupervised discovery of topics in unlabeled data may be used to improve the performance of various kinds of classifiers (such as sentiment analysis) and natural language processing applications. Being unsupervised is both a blessing and a curse. It is a blessing because good labeled data is a scarce resource, so improving tools that depend on labeled data by extracting knowledge from the vast amounts of unlabeled data is very useful. It is a curse because the methods used to discover topics are generally computationally intensive.
A topic model—which is a probabilistic model for unlabeled data—may be used for the automatic and unsupervised discovery of topics in unlabeled data, such as a set of textual documents. Such a topic model is designed with the underlying assumption that words belong to sets of topics, where a topic is a set of words. For example, given a set of scientific papers, a topic model can be used to discover words that occur together (and therefore form a topic). One topic could include words such as “neuroscience” and “synapse”, while another topic could include words such as “graviton” and “boson”.
Topic models have many applications in natural language processing. For example, topic modeling can be a key part of text analytics such as Name Entity Recognition, Part-of-Speech Tagging, retrieval of information for search engines, etc. Unfortunately, topic modeling is generally computationally expensive, and it often needs to be applied on significant amounts of data, sometimes under time constraints.
The latest editions of GPUs have considerable computational potential, with even more potential for computational power. However, running topic modeling tasks on a GPU is challenging because GPUs expose a computational model that is very different from ordinary CPUs (e.g., processors and multicores). As such, algorithms that work well on ordinary CPUs need to be re-designed to be data-parallel for GPUs. Also, GPUs have small amounts of memory, and with enough parallelism, memory bandwidth becomes a bottleneck.
As such, it would be beneficial to implement a topic modeling algorithm that is highly data-parallel, and effectively manages memory issues in order to more efficiently create and sample topic models on a GPU.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.